Reinforcement learning (RL) can automate a wide variety of robotic skills,but learning each new skill requires considerable real-world data collectionand manual representation engineering to design policy classes or features.Using deep reinforcement learning to train general purpose neural networkpolicies alleviates some of the burden of manual representation engineering byusing expressive policy classes, but exacerbates the challenge of datacollection, since such methods tend to be less efficient than RL withlow-dimensional, hand-designed representations. Transfer learning can mitigatethis problem by enabling us to transfer information from one skill to anotherand even from one robot to another. We show that neural network policies can bedecomposed into "task-specific" and "robot-specific" modules, where thetask-specific modules are shared across robots, and the robot-specific modulesare shared across all tasks on that robot. This allows for sharing taskinformation, such as perception, between robots and sharing robot information,such as dynamics and kinematics, between tasks. We exploit this decompositionto train mix-and-match modules that can solve new robot-task combinations thatwere not seen during training. Using a novel neural network architecture, wedemonstrate the effectiveness of our transfer method for enabling zero-shotgeneralization with a variety of robots and tasks in simulation for both visualand non-visual tasks.
展开▼
